Interpretable Neural Networks with BP-SOM

Interpretation of models induced by artiicial neural networks is often a diicult task. In this paper we focus on a relatively novel neural network architecture and learning algorithm, bp-som, that ooers possibilities to overcome this diiculty. It is shown that networks trained with bp-som show interesting regularities, in that hidden-unit activations become restricted to discrete values, and that the som part can be exploited for automatic rule extraction.